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SECTION A 
PROJECT MANAGEMENT 


Al VERIFICATION TEST ORGANIZATION 


The verification test will be conducted under the auspices of the U.S. Environmental 
Protection Agency (EPA) through the Environmental Technology Verification (ETV) Program. 
It will be performed by Battelle, which is serving as the verification organization under the 
Environmental and Sustainable Technology Evaluations (ESTE) arm of ETV. 

The day to day operations of this verification test will be coordinated and supervised by 
Battelle, with the participation of the vendors who will be having the performance of their 
technologies for detecting lead in paint verified. Testing will be conducted at Battelle in 
Columbus, Ohio. Each vendor will provide Battelle with their respective technology and will 
train the Battelle staff in their technology use. Battelle technical staff as well as non-technical 
operators will operate the technologies during verification testing. 

The organization chart in Figure | identifies the responsibilities of the organizations and 
individuals associated with the verification test. Roles and responsibilities are defined further 
below. Quality Assurance (QA) oversight will be provided by the Battelle Quality Manager and 
also by the EPA Quality Manager, at her discretion. 
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Battelle 
Management 


Julius Enriquez 
EPA ESTE 
Project Officer 
Michelle Henderson 
Zachary Willenberg EPA 
Battelle Quality Manager 
Quality Manager 





Dr. Stephanie Buehler Lead Paint Test Kit 
Battelle Vendor 

[Seeeaciaceenoes Verification Representatives 

Test Coordinator 









Battelle 
Technical Staff 


Figure 1. Organization Chart for the Verification Test 
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Al.1_ Battelle 


Dr. Stephanie Buehler is Battelle's Verification Test Coordinator for this test. In this role, 
Dr. Buehler will have overall responsibility for ensuring that the technical, schedule, and cost 
goals established for the verification test are met. Specifically, Dr. Buehler will: 

e Prepare the draft test/QA plan, verification reports, and verification statements. 

e Establish a budget for the verification test and manage staff to ensure the budget is 

not exceeded. 

e Revise the draft test/QA plan, verification reports, and verification statements in 
response to reviewers’ comments. 

e Assemble a team of qualified technical staff to conduct the verification test. 

e Direct the team in performing the verification test in accordance with this test/QA 
plan. 

e Hold a kick-off meeting approximately one week prior to the start of the verification 
test to review the critical logistical, technical, and administrative aspects of the 
verification test. Responsibility for each aspect of the verification test will be 
confirmed. 

e Serve as the primary point of contact for vendor representatives. 

e Ensure that confidentiality of sensitive vendor information is maintained. 

e Assist vendors as needed during verification testing. 

e Become familiar with the operation and maintenance of the technologies through 
instruction by the vendors, if needed. 

e Review and approve internal QA reviews and assessment reports. 

e Review independent QA document reviews and assessment reports by EPA quality 
manager. 

e Respond to any issues raised in assessment reports, audits, or from test staff 
observations, and institute corrective action as necessary. 

e Coordinate distribution of the final test/QA plan, verification reports, and verification 


statements. 
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Technical staff from Battelle will support Dr. Buehler in planning and conducting the 


verification test. The responsibilities of the technical staff will be to: 
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e Assist in planning for the test, and making arrangements for the receipt of and 
training on the technologies. 

e Attend the verification test kick-off meeting. 

e Assist vendor staff as needed during technology receipt and training. 

e Conduct verification testing using the vendor’s technology and per the final test/QA 
plan. 

e Conduct reference testing. 

e Perform statistical calculations specified in this test/QA plan on the technology data 
as needed. 

e Provide results of statistical calculations and associated discussion for the verification 
reports as needed. 

e Support Dr. Buehler in responding to any issues raised in assessment reports and 


audits related to statistics and data reduction as needed. 


Mr. Zachary Willenberg is Battelle’s Quality Manager. Mr. Willenberg will: 

e Review and approve the draft and final test/QA plan. 

e Attend the verification test kick-off meeting. 

e Conduct a technical systems audit at least once during the verification test, or 
designate other QA staff to conduct the audit. 

e Audit at least 10% of the verification data or designate other QA staff to conduct the 
data audit. 

e Prepare and distribute an assessment report for each audit. 

e Verify implementation of any necessary corrective action. 

e Request that Battelle’s Verification Test Coordinator issue a stop work order if audits 
indicate that data quality is being compromised. 

e Provide a summary of the QA/QC activities and results for the verification reports. 

e Review and approve the draft and final verification reports and verification 


statements. 
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Al.2 Technology Vendors 


The responsibilities of the technology vendors are as follows: 
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Review and provide comments on the draft test/QA plan. 

Accept (by signature of a company representative) the final test/QA plan prior to test 
initiation. 

Provide adequate units of their technology for evaluation during the verification test. 
Provide all other equipment/supplies/reagents/consumables needed to operate their 
technology for the duration of the verification test. 

Supply training on the use of the technology, and provide written consent and 
instructions for test staff to carry out verification testing, including written 
instructions for routine operation of their technology. 

Provide maintenance and repair support for their technology, on-site if necessary, 
throughout the duration of the verification test. 

Review and provide comments on the draft verification report and statement for their 


respective technology. 


A1.3. EPA 


EPA’s responsibilities are based on the requirements stated in the “Environmental 


Technology Verification Program Quality Management Plan” (EPA QMP).' The roles of specific 
EPA staff are as follows: 


Ms. Michelle Henderson is EPA’s Quality Manager for the verification test. 


Ms. Henderson will: 


Review the draft test/QA plan. 

Approve the final test/QA plan. 

Perform at her option one external technical systems audit during the verification test. 
Notify the EPA ESTE Project Officer of the need for a stop work order if the external 
audit indicates that data quality is being compromised. 

Prepare and distribute an assessment report summarizing results of the external audit. 
Review draft verification reports and verification statements. 


Approve final verification reports and statements. 


Mr. Julius Enriquez is EPA’s ESTE Project Officer. Mr. Enriquez will: 


Review the draft test/QA plan. 
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e Approve the final test/QA plan. 

e Review independent QA document reviews and assessment reports by EPA quality 
manager. 

e Review Battelle QA reviews and assessment reports and initiate corrective actions. 

e Review the draft verification reports and verification statements. 

e Oversee the EPA review process for the test/QA plan, verification reports, and 
verification statements. 

e Approve final verification reports and statements. 

e Coordinate the submission of verification reports and verification statements for 


signature by laboratory director and posting on the ETV website. 


A1.4 Subcontract Laboratory 


Any laboratory providing reference measurements will follow the requirements of the 
reference methods as well as the QC requirements as stated in this test/QA plan. A subcontract 
laboratory will provide reference measurements for the paint chip samples from each PEM. The 
responsibilities of this laboratory will include: 

e Proper receipt and handling of sample material. 

e Accurate measurement of the target analyte(s) or target parameter(s). 

e Submission of data and any supporting documents to Battelle. 

e Participation in audit by Battelle Quality Manager and/or EPA’s Quality Manager, if 

requested. 

e Submission of QC limits/criteria used by the laboratory for inclusion in this 


document. 


A2 BACKGROUND 


The ETV Program conducts third-party performance testing of commercially available 
technologies. The purpose of ETV is to provide objective and quality assured performance data 


on environmental technologies, so that users, developers, regulators, and consultants can make 
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informed decisions about purchasing and applying these technologies. Stakeholder committees 


of buyers and users of such technologies provide input on technology verifications. 
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Lead-based paints were commonly used in houses in both interior and exterior 


applications prior to 1978, when the US government banned the use of lead-based paint in 
residential applications. The term lead-based paint means paint or other surface coatings that 
contain lead at contents that equal or exceed a level of 1.0 milligrams per centimeter squared 
(mg/cm”) or 0.5 percent by weight. This paint still exists in many of these houses across the 
country. The accurate and efficient identification of lead-based paint in housing is important to 
the Federal government as well as private individuals living in residences containing such paints. 
Renovation, Repair, and Painting (RRP) activities may disturb painted surfaces and produce a 
lead exposure hazard. Such disturbances can be especially harmful to children and pregnant 
women as lead exposure can cause neurological and developmental problems in both children 
and fetuses. In fact, because of the large amount of pre-1978 housing stock, a report by the 
President’s Task Force on Environmental Health Risks and Safety Risks to Children found that 


approximately 24 million US dwellings were at risk for lead-based paint hazards’. 


A3  ~=VERIFICATION TEST DESCRIPTION AND SCHEDULE 
A3.1 Summary of Technology Category 


There are lead-based paint test kits available to help home owners and contractors 
identify lead-based paint hazards before any RRP activities take place so that proper health and 
safety measures can be enacted. However, many of these test kits have been found to have high 
rates of false positives’. The Renovation, Repair, and Painting rule’ calls for an EPA evaluation 
and recognition program for test kits that are candidates to meet the goal of a 5% false negative 
rate and 10% false positive rate. As stated in the Preamble to the rule, the test kit performance 
must be validated by a laboratory independent of the kit manufacturer, using ASTM 
International’s E1828, Standard Practice for Evaluating the Performance Characteristics of 
Qualitative Chemical Spot Test Kits for Lead in Paint? or an equivalent validation method. EPA 
will then only recognize those kits that have been verified through this process. ETV will 
coordinate the testing and supply the data that will be used in the recognition process. This plan 


incorporates ASTM Method E1828 guidelines’. 
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A3.2 Verification Test Schedule 


Table 1 shows the planned schedule of testing and data analysis/reporting activities to be 
conducted in this verification test. The planned dates for conducting verification tests of lead 
paint test kits are December 2009 - April 2010 at Battelle’s laboratories in Columbus, Ohio. A 
final verification test schedule with specific test dates will be provided to participating vendors 
once those details are known. It will be necessary for participating vendors to provide their 
technologies to Battelle by the specified date so testing staff may become familiar with operating 
the kits before testing begins. Vendor staff will provide training in operating the technologies 
either in person or by teleconference. The period of operation for verification testing will be 
approximately four to six weeks. The test procedures are described in Section B of this test/QA 
plan. 

Subsequent to the verification test, a separate verification report will be drafted for each 
participating technology. These reports will be reviewed by the respective vendor and by peer 
reviewers, and submitted to EPA for final signature and subsequent publication. Technologies 
and associated equipment (but not consumables) will be returned to the vendors at the 


completion of report writing, unless other arrangements have been made with Battelle. 


Table 1. Planned Verification Test Schedule 


Dates Testing Activities Data Analysis and Reporting 
December 2009 | Training of verification testing staff on 
technology use 


December 2009 | Conduct verification testing Review and compile test data and records as they 
— April 2010 become available. 

Review and summarize verification testing staff 
observations. 

Prepare report templates and complete common 
sections of reports. 

Evaluate and analyze data generated during testing 


May — July 2010 Complete draft reports and submit for vendor, EPA, 
and peer reviews. 





February - May 
2010 


July — Revise draft reports and submit final reports for 
September 2010 EPA approval. 


A3.3 Test Site 
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Testing will be conducted in Battelle laboratories in Columbus, Ohio. There will be no 


field testing, i.e., testing at an offsite location outside of the laboratory, such as a house, 
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conducted during this technology verification. EPA is considering the possibility of a future 


verification test involving real-world field environments. 


A3.4 Health and Safety 


Battelle will conduct all verification testing and reference lead paint spot test kit 
measurements following the safety and health protocols in place for the laboratory and facilities. 
This includes maintaining a safe work environment and a current awareness of handling 
potentially toxic chemicals. Exposure to potentially toxic chemicals will be minimized, personal 
protective equipment will be worn, and safe laboratory practices will be followed, as necessary. 
Health and Safety will be reviewed with Battelle’s Safety Officer once the specific technologies 


to participate in the test are known. 


A4 QUALITY OBJECTIVES AND CRITERIA FOR MEASUREMENT DATA 


In performing the verification test, Battelle will follow the technical and QA procedures 
specified in this test/QA plan and will comply with the data quality requirements in the EPA- 
approved QMP* for the Advanced Monitoring Systems (AMS) Center, except where differences 
are noted for ESTE per the EPA ETV Program QMP.' The objective of this verification test is to 
evaluate the performance of test kits for the detection of lead in paint. This evaluation will assess 
the capabilities of the lead paint spot test kits against laboratory prepared performance evaluation 
material (PEM) samples, and will include a comparison between the lead paint test kit results and 
those of a standard technique as described in Section B4. Additionally, this verification test will 
rely upon verification testing staff observations to assess other performance characteristics of the 
lead paint test kits. Only qualitative results (e.g., detect/non-detect of lead at specified levels) 
will be considered for each technology. Below is a discussion of the quality objectives and the 


criteria for measurement data that have been established to assure that the test objectives are met. 


A4.1 Quality Objectives 


Data quality objectives indicate the minimum data quality required to meet the lead paint 
spot test kits verification objectives. Data quality objectives for this verification test include 


those related to the reference method performance and those related to the test kit performance. 
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Data quality objectives for the reference method (see Section B4) are presented in terms of data 
quality indicator (DQJ) criteria for the critical measurements associated with the reference 
method and are listed in Table 2 and discussed in Section A4.2. The quality of the reference 
measurements will be monitored using QC samples and procedures, as described in the testing 
laboratory’s procedures or the method. These requirements are further discussed in Section B. 
Method blanks, positive control samples, and negative control samples are expected to be 
included as QC samples for each technology. Method blank samples will include performance 
evaluation materials (PEMs) with 0.0 mg/cm” lead paint as well as each PEM substrate (wood, 
metal, drywall, and plaster) with no paint. Positive and negative control samples, if provided 
with a test kit, will be analyzed according to the kit’s instructions. 

The EPA Quality Manager will perform a technical systems audit (TSA) of the 
subcontract laboratory conducting reference analyses. The Battelle Quality Manager or his 
designee will perform a TSA of the actual evaluation of the test kits at least once during this 
verification test and will audit at least 10% of the verification data acquired, including the data 
packages received from the subcontract reference laboratory. The EPA Quality Manager also 


may conduct an independent TSA of the verification test, at her discretion. 


A4.2 Criteria for Measurement Data 


Reference measurements will be conducted by an NLLAP-accredited laboratory using 
inductively coupled plasma-atomic emission spectrometry (ICP-AES) on paint chip samples 
from each PEM used in the verification test. Table 2 presents the minimum DQIs and criteria for 
the reference method critical measurements. These DQIs and criteria are based on NLLAP 
guidelines and are consist with the selected NLLAP-accredited laboratory’s criteria and 
procedures. The method detection limit for the reference analyses will be determined based on 
the criteria provided by the subcontract laboratory selected to perform the reference analyses. 
The reference method measurement quality will be assured by adherence to these DQI criteria. 
For batches of less than 20 samples, at least one sample will be analyzed for each applicable 
DQI. Recommendations for appropriate positive and negative controls and their critical 


measurements for the lead paint spot test kits will be provided by each vendor, as appropriate. 
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Documentation of training related to technology testing, data analysis, and reporting is 


maintained for all Battelle technical staff in training files at their respective Battelle location. The 


Battelle Quality Manager may verify the presence of appropriate training records prior to the 


start of testing. The technology vendor will be required to train Battelle technical staff in the 


operation of his/her technology prior to the start of testing. Battelle will document this training 


with a consent form, signed by the vendor, which states which specific Battelle staff have been 


trained and determined by the vendor to be competent in operation of the vendor’s technology. 


Table 2. DQIs and Criteria for Critical Measurements for Reference Method 











Method of Minimum 
DQI Pees Frequency Acceptance Corrective Action 
Assessment kaa 
Criteria 
Precision Replicate (duplicate) One per 20 Within +25% relative | Flag data; reanalyze QC 
analyses of test sample | samples or batch | percent difference samples; if these QC 
extract (min. 5% (RPD) samples are out of range, 
frequency) then repeat entire analysis 
including recalibration 
and all QC samples 
Bias and Accuracy of | Instrument Verified daily or | Per most stringent Recalibrate instrument 


Instrument 


calibration/performance 
verification using 
matrix-matched 
reference standard 
materials of the same 
matrix as the samples 


prior to analyzing 
samples 


instrument, 


laboratory, or method 


guidelines 





Bias and Accuracy of 
Sample 
Measurements 


Independent 
Calibration Verification 
(ICV) — lead standard 
at concentration in the 
range of lead levels 
tested 


Once per day 
after calibration 


Within +10% of 
known value 





Recalibrate instrument 








Initial Calibration 
Blank (ICB) - contains 
no lead and is used for 
initial calibration and 
zeroing instrument 
response. The ICB 
must be matrix 
matched to acid content 
present in sample 
digestates. 





Once per run at 
the beginning of 
the run 





Absolute value not 


more than 20% of the 
regulatory limit or the 


level of concern 





Prepare new calibration 
curves 
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Method of Naunepaeae 
DQI Frequency Acceptance Corrective Action 
Assessment cae 
Criteria 
Continuing Calibration | At the beginning | Within +20% of Establish new calibration 
Verification (CCV)-— | and end ofa known value curve and reanalyze 


independent reference 
standards or the same 
standards used to 
prepare the instrument 
calibration curve 


sample run, as 
well as every 12 
hours, or 
according to 
instrument 
manufacturer’s 
recommendations, 
or according to 
instrument 
Performance 
Characteristic 
Sheet, or at a 
predetermined 
SOP frequency 
(once every 10 
samples), 
whichever is 
more frequent 


samples; sample analysis 
shall not continue or be 
restarted until a new 
calibration curve is 
established and verified 





Interference Check 
Sample (ICS) - A 
standard solution (or 
set of solutions) used to 
verify accurate analyte 
response in the 
presence of possible 
interferences from 
other analytes present 
in samples. The ICS 
must be matrix 
matched to the reagent 
content present in 
sample digestates. 


At the beginning 
and end of each 
run or twice every 
12 hours 


Within 20% of known 


value 


Apply correction factors 
to sample results as 
appropriate 














Continuing Calibration | After each ICS Absolute value not Flag data; attempt to 
Blank (CCB) - A and CCV more than 20% of the | determine source of 
standard solution which regulatory limit or contamination; reanalyze 
has no lead and is used level of concern QC samples; if these QC 
to verify blank samples are out of range, 
response and freedom then repeat entire analysis 
from carryover including recalibration 
and all QC samples 
Laboratory Control 1 per 10-20 Within +20% of Flag data; reanalyze QC 


Sample (LCS) — same 
matrix as test samples 
with lead concentration 
near the level of 
concern or regulatory 
level; wherever 
possible shall not 
require extensive 
pretreatment dilution or 
concentration prior to 





samples or batch 
(minimum 5%) 





known value 





samples; if these QC 
samples are out of range, 
then repeat entire analysis 
including recalibration 
and all QC samples 
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Method of Minimum 
DQI eanone Frequency Acceptance Corrective Action 
Assessment ete 
Criteria 

analysis; shall be either 
NIST Standard 
Reference Materials or 
commercially available 
certified reference 
materials 
Matrix Spike Sample — |1 per 20 samples | Within +25% of Flag data; reanalyze QC 
prepared using split or batch calculated value samples; if these QC 
sample (before any (minimum 5%) samples are out of range, 
digestion when then repeat entire analysis 
possible); lead level including recalibration 
spiked shall be enough and all QC samples 


to result in final lead 
concentration of the 
prepared sample of 5x 
the sample’s observed 
lead concentration, or 
5x the method 
detection limit, 
whichever is greater 








Duplicate Samples — of | 1 per 20 samples | Within +25% of RPD | Flag data; reanalyze QC 
test sample extract only | or batch samples; if these QC 
(minimum 5%) samples are out of range, 
then repeat entire analysis 
including recalibration 

















and all QC samples 
Method Blank — 1 per 20 samples | Absolute value not Flag data; attempt to 
mixture of all reagents | or batch more than 20% of the | determine source of 
used for digestion but |(minimum5%) | regulatory limit or contamination; reanalyze 
without the matrix; is level of concern QC samples; if these QC 
carried through all samples are out of range, 
steps of the analysis then repeat entire analysis 
starting with digestion including recalibration 

and all QC samples 


A6 DOCUMENTATION AND RECORDS 


The records for this verification test include the test/QA plan, the protocols, laboratory 
record books (LRB), data collection forms, electronic files (both raw data and spreadsheets), and 
the final verification report and statement. All of these records will be maintained in the 
Verification Test Coordinator’s office or at the testing locations during the test and will be 
transferred to permanent storage at Battelle’s Records Management Office at the conclusion of 


the verification test. All Battelle LRBs are stored indefinitely, either by the Verification Test 
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Verification of Qualitative Spot Test Kits for Lead in Paint 
Test/QA Plan 

Page 20 of 67 

Version DRAFT 

Date: 3/30/2010 


the reference measurements made by the subcontractor laboratory will be submitted to Battelle 
after making the measurement and obtaining the results of the analyses. Table 3 provides further 
details the data recording practices and responsibilities. QA documents generated over the course 
of this verification test include audit and assessment reports and will be maintained by the 
Battelle Quality Manager. Copies of audit and assessment reports will be downloaded into the 
ETV web database so that EPA may access it if needed. 

All written records must be in ink. Any corrections to notebook entries, or changes in 
recorded data, must be made with a single line through the original entry. The correction is then 
to be entered, initialed, and dated by the person making the correction. In all cases, strict 
confidentiality of the raw data from each vendor’s technology, and separation of data from 
different technologies, will be maintained throughout the verification test. Separate files 
(including manual records, printouts, and/or electronic data files) will be kept for each 


instrument. 


Table 3. Summary of Data Recording Process 


Recorded Recorded Data 
Dates, time, and LRBs or data Start/End of test Battelle Used to organize and 
details of test events | recording forms procedure, and at check test results; 
each change of test manually 
parameter incorporated 


in data spreadsheets 
as necessary 


Sample (PEMs) used | LRBs or data When each PEM is _ | Battelle Incorporated into 
(IDs, dates, etc.) recording forms used, throughout test verification report as 
duration necessary 


Test kit procedures _| Data sheets and LRB | Throughout test Battelle Manually 

and sample results duration incorporated into 
data spreadsheets for 
statistical analysis 
and comparisons 





Reference method Throughout sample | Battelle or Used to demonstrate 
sample preparation preparation subcontract validity of samples 
laboratory submitted for 
reference 
measurements 
Reference method LRB or data Throughout sampling | NLLAP-accredited | Retained as 
procedures, recording forms and analysis laboratory documentation of 
calibrations, QA, etc. processes reference method 
performance 
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SECTION B 
MEASUREMENT AND DATA ACQUISITION 


Bl EXPERIMENTAL DESIGN 


This verification test will specifically address verification of spot test kits for the 
detection of lead in paint. This test follows procedures described in ASTM E1828°. The lead 
paint test kits will be tested only in a laboratory under controlled conditions; no field testing will 
take place during this verification test. This will allow comparison of the technology results to a 
reference method using a specified set of performance evaluation materials (PEMs). PEMs will 
be 3 inch by 3 inch squares of wood, metal, plaster, or drywall coated with paint of various 
colors containing a range of lead concentrations. PEM samples will be analyzed in at least 
duplicate by the test kits and also analyzed by the reference method (ICP-AES). The lead paint 
test kits participating in this test will be evaluated based on qualitative results, indicating only the 
presence or absence of lead in the paint at specified concentrations (see Section B1.1). Some test 
kits may provide quantitative results. In the instance where quantitative measures are used in 
determining the results for a particular technology, a qualitative result will be reported (i.e., 
presence or absence of the contaminant of interest) as with the other technologies, and the 
quantitative measure used to determine that result will also be reported for that sample but will 
not be used in any other data analyses as described in Section B1.2. The performance of the lead 
paint test kits will be verified based on sensitivity, precision, false positive/negative rates, matrix 
effects, and operational factors. These parameters are discussed in detail in Section B1.1 and 
B1.2. 

The analyses will be performed according to the vendor’s recommended procedures as 
described in the user’s instructions or manual, which should be consistent with training provided 
to Battelle staff. Similarly, calibration and maintenance of the technologies will be performed as 
specified by the vendor. Results from the technologies being verified will be recorded manually 
by the operator on appropriate data sheets or captured in an electronic data system and then 
transferred manually or electronically for further data workup. Qualitative characteristics of each 
technology such as ease of use will be assessed through observations made by the Test 
Coordinator and operators throughout the verification test. The results from each technology will 


be reported individually. No direct comparison will be made between technologies, but each 
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technology will undergo similar testing so it is convenient for end users to evaluate the ETV 


testing results. 


Bl1.1 Test Procedures 


Qualitative spot test kits for lead in paint will be evaluated against a range of lead 
concentrations in paint on various substrates through the use of PEMs. PEMs are 3 inch by 3 
inch square panels of wood (pine and poplar), metal, drywall, or plaster that will be prepared by 
Battelle’. Table 4 shows the PEMs to be tested for each test kit. Each PEM will be coated with 
the same thickness of either white lead or lead chromate paint. The paint will contain lead at 0.4, 
0.6, 1.0, 1.4, 2.0, and 6.0 mg/cm’. These lead concentrations were chosen based on guidelines 
provided in EPA’s lead RRP rule* as well as to represent potential lead levels in homes. 

After production of the 6.0 mg/cm? PEMs, reference analyses results indicated that actual 
lead levels for these PEMs were outside of the anticipated 6.0 mg/cm” target level. For lead 
chromate PEMs at 6.0 mg/cm”, 18 reference panels dispersed throughout the PEMs during 
production’ indicated that actual concentrations ranged from 4.8 - 6.2 mg/cm? with a mean of 5.2 
mg/cm? and a CoV of 9.6%. The 18 reference panels coated during the production of 6.0 
mg/cm” PEMs using white lead indicated that actual concentrations ranged from 5.6 to 18.4 
mg/cm’. Four white lead reference PEMs were considered to be “outliers” (18.4, 12.8, 11.3, and 
11.1 mg/cm’). Excluding these panels results in an average concentration of 8.0 mg/cm? with a 
CoV of 12% for this level. Though both the lead chromate and white lead PEMs are outside of 
the expected 6.0 mg/cm’ concentration, it was determined that these PEMs were acceptable for 
use. The purpose of the 6.0 mg/cm? lead level is to evaluate the test kit’s response at a level well 
above the action level of 1.0 mg/cm’. Both the lead chromate and white lead PEMs satisfy this 
need and will be used in this verification test. Some modifications will be made, however, for 
the use of the white lead PEMs at this level. Because there were four reference panels with quite 
high lead levels, PEMs produced within the range of these four reference panels, and thus 
thought to have the same lead levels, will be distributed evenly amongst all participating test kits, 
to the extent possible. When analyzing the results from this lead level, consideration will be 
given to conducting analyses with all data for this lead level as well as without data from PEMs 
with >10 mg/cm”. 

Paint containing no lead (0.0 mg/cm”) will also be applied to each substrate and tested. 


Two different layers of paint will be applied over the leaded paint. One will be a primer 
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designed for adhesion to linseed oil-based paint and the second coat will be a typical interior 
modern latex paint tinted to one of three colors: white, red-orange, or grey-black. These colors 
were chosen by EPA based on the potential of certain colors to interfere or not with lead paint 
test kit operations. The top-coat paint manufacturers’ recommended application thickness will 
be used. Two coats at the recommended thickness will be applied. Each substrate will be tested 
without paint, in the same manner as all other PEMs (1.e., per the test kit instructions), to 
determine if the substrate material itself is causing any effects on the performance of the test kits. 
Two unpainted PEMs of each substrate will be evaluated using each test kit. 

Each spot test kit for lead paint will be operated by a technical operator. This operator 
will be a Battelle staff member with laboratory experience. The technical operator will be 
trained by the vendor in the operation of the vendor’s test kit. The same technical operator will 
operate a given test kit throughout the course of testing. Multiple technical operators may 
operate different lead paint test kits. Because these test kits are anticipated to be used by 
certified remodelers, renovators and painters, the test kits will also be evaluated by a non- 
technical operator depending on the operational and potential safety issues surrounding each test 
kit. Because this verification test will involve the evaluation of lead-based paint, any disturbance 
of that paint could pose a potential health hazard. Ifa test kit’s operation does involve disturbing 
the paint on the PEM and thus pose a health risk to the operator, then it may not be feasible to 
evaluate that test kit using a non-technical operator. In such an instance, any technical operator 
from Battelle would have to undergo specific health and safety training to operate the test kit and 
all appropriate health and safety practices would have to be followed during testing. Ifa non- 
technical operator is used, the non-technical operator will be a certified renovator with little to no 
experience with lead. The non-technical operator will be trained in the use of the lead paint test 
kit by a Battelle staff person who has experience operating test kits in general, but not by the 
technical operator who was trained by the vendor. This scenario will approximate the training 


renovators are expected to receive under the RRP rule’, 
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Table 4. PEMs Testing Scheme for Each Test Kit. 








Weadiltevel PEMs Analyzed Per Test Kit by Topcoat Color 


Lead Type (mg/cm’) Substrate Red-Orange | Grey-Black Total 
Wood 3 3 
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Tests will be performed in at least duplicate on each PEM by each operator, technical and 
non-technical, depending on available space and test kit operation requirements. Replicates will 
be tested in succession by each operator on a given PEM. PEMs will be analyzed blindly by 
each operator in that the PEMs used for analysis will be marked with a non-identifying number. 
Test kit operators will not be made aware of the paint type, lead level, or substrate of the PEM 
being tested. PEMs will be tested in no particular order. 

Paint chip samples from each PEM will be analyzed by a National Lead Laboratory 
Accreditation Program (NLLAP) accredited laboratory using ICP-AES to confirm the lead level 
of each PEM used for testing. The paint chip samples for reference analyses will be collected by 
Battelle according to ASTM E1729*. The reference analyses will confirm the lead level of each 
PEM to ensure an accurate understanding of each test kit’s performance. Lead levels determined 
through the reference analysis will be used for reporting and statistical analyses. 


The technologies will be evaluated for the following parameters: 


B1.1.1 False Positive and False Negative Rates 

A false positive response will be defined as a positive result when regulated lead-based 
paint is not present. For this test, false positive rates will be assessed on panels with target lead 
levels at 0.6 mg/cm’ and lower. Per the guidelines set forth in EPA’s April 22, 2008 RRP rule’, 
panels with an ICP-AES confirmed lead level greater than 0.8 mg/cm” will not be used in the 
false positive analysis. 

A false negative response will be defined as a negative response when regulated lead- 
based paint is present. For this test, false negative rates will be assessed on panels with target 
lead levels at 1.4 mg/cm? and higher. Per the guidelines set forth in EPA’s April 22, 2008 RRP 
rule*, panels with an ICP-AES confirmed lead level less than 1.2 mg/cm’ will not be used in the 
false negative analysis 

False positive and negative rates will be grouped by paint type (lead chromate vs. white 
lead). Results will also be grouped across paint types by PEM substrate and by color. Results 
will also be examined by operator type (i.e., technical vs. non-technical). 

Based on stakeholder input, the EPA lead paint action level of 1.0 mg/cm” lead was 
included for analysis as part of the verification test. Though evaluations of test kit performance 


based on this level is not within the guidelines of the EPA RRP rule’, false positive and negative 
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rates, in addition to those stated above, will also be calculated for each test kit based on 1.0 
mg/cm” lead. Thus, false positive rates will be assessed on panels with lead levels at 1.0 mg/cm? 
or lower and false negative rates will be assessed on panels with lead levels at 1.0 mg/cm” or 
higher. For panels that measure 1.0 mg/cm”, positive results will be considered “correct” and 
negative results will be considered false negative. If the lead concentration of the PEM is 
actually greater than 1.0 mg/cm’ (e.g., 1.1 mg/cm’), then negative results will be considered false 
negatives. If the lead concentration of the PEM is actually less than 1.0 mg/cm? (e.g., 0.9 


mg/cm’), then positive results will be considered false positives. 


B1.1.2 Precision 
Precision will be measured by the reproducibility of responses for replicate samples 
within a group of PEMs. Groups of PEMs to be evaluated for precision will include lead 


concentrations and substrate material at a specific lead concentration. 


B1.1.3 Sensitivity 
The sensitivity or lowest detectable lead level will be identified based on the detection 


results across all PEM lead levels. 


B1.1.4 Modeled Probability of Test Kit Response 
Logistic regression models will be used to determine the probabilities of positive or 
negative responses of the test kit at the 95% confidence level, as a function of lead concentration 


and other covariates, such as substrate type. 


B1.1.5 Matrix Effects 

Covariate adjusted logistic regression models will be used to determine whether any of 
the PEMs parameters (color, substrate, etc) affects the performance of the test kit. Type III 
Statistics and comparison of Likelihoods from logistics regression models will be used to 


determine the statistical significance of these factors. 


B1.1.9 Operational Factors 

Operational factors such as ease of use, operator bias, and helpfulness of manuals, will be 
evaluated based on Operator and Verification Test Coordinator observations. Sustainability 
metrics such as volume and type of waste generated from the use of each test kit, toxicity of the 


chemicals used, and energy consumption will also be evaluated. These metrics will be discussed 
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by detailing how much waste is generated and what the waste is composed of, providing 
information on how the waste should be properly handled, presenting a summary of the pertinent 
MSDS information, when available, and noting whether the test kit used batteries, a power 
supply, or no energy source is needed. Information on how many tests each kit can perform as 


well as the shelf life of the test kit and chemicals used as part of the test kit will also be reported. 


B1.2 Statistical Analysis 


The statistical methods and calculations used for evaluating quantitative performance 
parameters are described in the following sections. ICP-AES reference analyses will confirm the 
lead level of each PEM to ensure an accurate understanding of each test kit’s performance. Lead 
levels determined through the reference analysis will be used for reporting and statistical 


analyses. 


B1.2.1 False Positive and False Negative Rates 

A false positive response will be defined as a detect from the lead paint test kit when 
evaluated on PEMs with target lead levels at and below 0.6 mg/cm? (i.e., 0, 0.4, and 0.6 mg/cm? 
levels). Per the guidelines set forth in EPA’s April 22, 2008 RRP rule’, panels with an ICP-AES 
confirmed lead level greater than 0.8 mg/cm” will not be used in the false positive analysis. A 
false negative response will be defined as a non-detect from the lead paint test kit when 
evaluated on PEMs with target lead levels at and above 1.4 mg/cm’ (i.e., 1.4, 2.0, 6.0 mg/cm”). 
Per the guidelines set forth in EPA’s April 22, 2008 RRP rule’, panels with an ICP-AES 
confirmed lead level less than 1.2 mg/cm? will not be used in the false negative analysis 

False positive and negative rates will be grouped by paint type (lead chromate vs. white 
lead). Results will also be grouped across paint types by PEM substrate and by color. Results 
will also be examined by operator type (1.e., technical vs. non-technical, where applicable). 

Based on stakeholder input, the EPA lead paint action level of 1.0 mg/cm” lead was 
included for analysis as part of the verification test. Though evaluations of test kit performance 
based on this level is not within the guidelines of the EPA RRP rule’, false positive and negative 
rates, in addition to those stated above, will also be calculated for each test kit based on 1.0 
mg/cm? lead. Thus, false positive rates will be assessed on panels with lead levels at 1.0 mg/cm? 
and lower and false negative rates will be assessed on panels with lead levels at 1.0 mg/cm? and 


higher. 
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False positive and negative rates will be evaluated as the number of positive or negative 
results, respectively, out of the total number of PEM samples evaluated without or with regulated 


lead-based paint, per the concentration levels stated above. 





# iti lt 
False Positive Rate = oo (1) 


total # of PEMs without regulated lead - based paint 


False Negative Rate = sa RESUME TESS : (2) 
total # of PEMs with regulated lead - based paint 





B1.2.2 Precision 

Precision will be measured by the reproducibility of responses for replicate samples 
within a group of PEMs. Groups of PEMs to be evaluated for precision will include individual 
lead concentration levels (e.g., all PEMs at 0.4 mg/cm’) and substrate material at a specific lead 
concentration (e.g., all metal PEMs at 1.4 mg/cm’). Responses will be considered inconsistent if 
25% or more of the replicates differ from the response of the other samples in the same group of 
PEMs. An overall precision for each test kit will be assessed by paint type by calculating the 
overall number of consistent responses for all the sample sets of either white lead or lead 
chromate-painted PEMs. The results will be reported as the percentage of consistent responses 


out of all replicate sets for those paint types (see Equation 3). 


# of consistent responses of replicate sets : 





Precision (“% consistent results) = 100 (3) 


total number of replicate sets 
B1.2.3 Sensitivity 
The sensitivity or lowest detectable lead level for each test kit will be identified based on 
the detection results across all PEM lead levels. The lowest PEM lead level with consistent 
positive or “detect” responses will be considered the lowest detectable level. The identified 


lowest detectable lead level will be reported and discussed. 


B1.2.4 Modeled Probability of Test Kit Response 


Logistic regression models will be used to determine the probabilities of positive or 
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negative responses of the test kit at the 95% confidence level, as a function of lead concentration 


and other covariates, such as substrate type. An evaluation of the bivariate relationship between 
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the response variable and each individual candidate explanatory variable will be performed by 
fitting single covariate logistic models to assess the predictive ability of each of the PEM 
parameters. Using the results from these bivariate analyses a parsimonious multivariate model 
will be developed including a set of explanatory variables which are most predictive of the 
probability of the test kit response variable. The potential logistic regression model will take the 


form as below: 


logit (Pr(Y, =1)= XB (4) 


where Y; is the outcome of the test kit, X; is a vector of explanatory variables associated with Y; 
and # represent a vector of unknown parameters which will be estimated with the model. 
Candidate independent variables associated with the response variable are operator type, lead 
type, lead level, substrate type, and topcoat color. Interactions between these predictor variables 
will also be assessed. Each level of the covariates can also be included using indicator 


variables. SAS procedures GENMOD or LOGISTIC will be used to fit the logistic model 


B1.2.5 Matrix Effect 

The covariate-adjusted logistic regression model described in section B1.2.4 will be used 
to assess the significance of PEM parameters and the interactions among them on the 
performance of the test kits. PEM parameters are included in the model as explanatory variables 
associated with the Yj; response variable. 

Comparison of the observed values of the response variable to predicted values obtained 
from models with and without the predictor variable in question will be the guiding principle in 


logistic regression model. The likelihood function is defined as 
L(B)=] [Qu - 7% us) ©) 
i=l 


where 7(Y,,,) is the conditional probability of Yjj=1 and [1—7(¥,,,)] 1s the conditional 


probability of Yjja=0 given the vector of explanatory variables (X). For purposes of assessing 
the significance of a group of p predictor variables (where p can be | or more), we compute the 


likelihood ratio test statistic, G, as follows: 
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G = -2 log, [likelihood without the p variables / likelihood with the p variables] (6) 


Under the null hypothesis, this test statistic will follow a chi-square distribution with p 
degrees of freedom. If the test statistic is greater than the 95" percentile of the chi-square 


distribution, then the group of variables, taken together, are statistically significant. 


B1.2.6 Operational Factors 

There are no statistical calculations applicable to operational factors. Operational factors 
such as ease of use, operator bias, average cost, average time for kit operation, and helpfulness of 
manuals, will be determined qualitatively based on Operator (both technical and non-technical) 
and Verification Test Coordinator observations. The non-technical operator will not receive any 
vendor support on the operation of the test kit throughout the test. Descriptions of observations 
made throughout testing will be reported and discussed. Sustainability metrics such as volume 
and type of waste generated from the use of each test kit, toxicity of the chemicals used, and 
energy consumption will be discussed. This discussion will be based on how much waste is 
generated and what the waste is composed of, information on how the waste should be properly 
handled, a summary of the pertinent MSDS information, when available, and noting whether the 
test kit used batteries, a power supply, or no energy source is needed. Information on how many 
tests each kit can perform as well as the shelf life of the test kit and chemicals used as part of the 


test kit will also be reported. 


B13 Reporting 


The data obtained in the verification test will be compiled separately for each vendor’s 
technology, and the data evaluations will be applied to each technology’s data set without 
reference to any other. At no time will data from different vendor’s technology be intercompared 
or ranked. Following completion of the data evaluations, a draft verification report and 
verification statement will be prepared for each vendor’s technology, stating the verification test 
procedures and documenting the performance observed. Each report will briefly describe the 
ETV Program and the procedures used in verification testing. The results of the verification test 
will then be stated, without comparison to any other technology tested, or comment on the 


acceptability of the technology’s performance. Each draft verification report will be submitted 
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for review by the respective technology vendor and by EPA and other peer reviewers. Comments 
on the draft report will be addressed in revisions of the report. The peer review comments and 
responses will be tabulated to document the peer review process. The reporting and review 
process will be conducted according to the requirements of the ETV QMP’. All final verification 
reports and statements will be made 508 compliant and will be posted on the ETV website 


(www.epa.gov/etv). 


B2 SAMPLING REQUIREMENTS 
B2.1 Sample Collection, Storage and Shipment 


PEM samples will be produced and stored prior to the beginning of the verification test 
and in accordance with the Revised Plan For Development And Production Of Performance 
Evaluation Materials For Testing Of Test Kits For Lead In Paint under the Environmental 
Technology Verification Program’. The film thickness, homogeneity, and lead levels of the 
paint applied to the PEMs will be verified prior to full-scale PEMs production via spray or draw 
down application on quality-controlled metal panels’. This process will test the paint 
formulation and application to ensure that the desired lead level can be achieved during full PEM 
production. PEMs will be measured for film thickness using a Positector 6000 coating thickness 
gauge. Subsequently, paint chip samples corresponding to the locations of the film thickness 
measurements will be obtained following ASTM E1729, Standard Practice for Field Collection 
of Dried Paint Samples for Subsequent Lead Determination.’ These samples will be shipped to 
the NLLAP-accredited laboratory for analysis. This process will not verify the lead levels of 
individual PEMs used for verification testing and will take place prior to the production of PEMs 
for this verification test. Details on this process can be found in the PEMs development plan’. 
The PEMs development plan, along with a summary of the homogeneity and lead level analyses, 
can be found in Appendix A. 

As part of the verification test, the lead level of paint from each PEM used in the 
verification test will be verified through ICP-AES analysis by an independent NLLAP-accredited 
laboratory. Paint chip samples from unused portions of each PEM will be collected by Battelle 
using guidelines set forth in ASTM E1729 Standard Practice for Field Collection of Dried Paint 


Samples for Subsequent Lead Determination.® Paint chips will be collected into small glass vials 
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according to a procedure to be prepared by Battelle (see Appendix B). This procedure will detail 
paint chip sampling guidelines for each substrate to ensure consistent paint chip collection 
throughout the verification test. These paint chips will be supplied to the subcontract laboratory 
for analysis. ICP-AES reference analyses will confirm the lead level of each PEM to ensure an 
accurate understanding of each test kit’s performance. Lead levels determined through the 
reference analysis will be used for reporting and statistical analyses. The lead concentrations 
expected across a batch of PEMs will be evaluated prior to their use in the ETV test through the 
use of reference PEMs during the production phase’. This process should help ensure that large 


deviations in concentrations do not exist across a particular lead level. 


B3 SAMPLE HANDLING AND CUSTODY REQUIREMENTS 


Sample custody will be documented for the shipping and analysis of paint chip samples 
to the subcontract laboratory using standard chain-of-custody (COC) forms provided by Battelle 
or supplied by the laboratory, as appropriate. Samples transferred within Battelle may be 
documented in bound sample login LRBs. Each COC form will summarize the analyses 
requested. The COC forms will track sample release from Battelle to the NLLAP laboratory. 
Each COC form will be signed by the person relinquishing the samples once that person has 
verified that the COC form is accurate. The original sample COC forms will accompany the 
samples; the shipper will keep a copy. Any discrepancies will be noted on the form and the 
sample receiver will immediately contact the Verification Test Coordinator to report missing, 
broken, or otherwise compromised samples. Copies of all COC forms will be delivered to the 


Verification Test Coordinator, and maintained with the test records. 


B4 LABORATORY REFERENCE METHOD 


Paint chips from an unused portion of each PEM will be analyzed by ICP-AES by an 
independent NLLAP-accredited laboratory. Paint chip samples will be collected by Battelle 
using guidelines set forth in ASTM E1729° and supplied to the subcontract laboratory for 
analysis. Because PEMs will be assumed to be homogenous, based on pre-production testing’, 
the specific place of paint chip collection will not matter, as any spot on the PEM should be 
representative of the entire panel. However, to the extent practicable, based on space needed for 


individual test kit operation on each PEM, varying places on each PEM will be selected for 
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collection of paint ships for reference analyses. The NLLAP-accredited laboratory will use 
ASTM 1645” or equivalent for paint digestion and EPA method 6010B"° or equivalent, along 
with their own laboratory SOPs for ICP-AES analysis. The subcontract laboratory will be 
responsible for providing calibrated instrumentation, performing all method QA/QC, and 
providing calibration records for any instrumentation used. ICP-AES reference analyses will 
confirm the lead level of each PEM to ensure an accurate understanding of each test kit’s 
performance. Lead levels confirmed through the reference analysis shall have a percent error of 
less than +15 percent of expected values. Reference measurements outside this range will be 


used in place of expected lead concentrations for reporting and statistical analyses. 


BS QUALITY CONTROL 


Steps will be taken to maintain the quality of data collected during this verification test. 
When confirmation analyses of the lead levels of the PEMs are performed, QC measures as 
noted in the subcontract laboratory’s operating procedures or the reference method and provided 
in Table 2 will be followed. The QC measures for the reference method will at least include the 
analysis of a method blank sample. Method blank samples will be analyzed to ensure that no 
sources of contamination are present. If the analysis of a method blank sample indicates a 
concentration above the minimum acceptance criteria provided in Table 2, contamination will be 
suspected. Any contamination source(s) will be corrected, and proper blank readings will be 
achieved, before proceeding with the analyses. A matrix spike sample as well as calibration 
verification standards will also be analyzed. Average acceptable recoveries for these samples are 
between 75-125%. Initial calibration standards will be run at the beginning of each set of 
analyses or at least once daily. The calibration coefficient must be at least 0.995. If this criteria 
is not met, the analysis will be stopped and recalibration will be performed. A continuing 
calibration verification will be run once every 10 samples. Duplicate samples will be run once 
every 10-20 samples. 

If quality control samples as provided with each lead paint test kit (e.g., positive/negative 
controls), then they will also be run per the vendor’s instructions. Painted PEMs containing no 
lead as well as each of the PEMs substrates containing no paint will also be run as part of the 


verification test. 
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B6é INSTRUMENT/EQUIPMENT TESTING, INSPECTION, AND MAINTENANCE 


The instruments used for the reference analyses will be tested and inspected as per the 
standard operating procedures or instrument manuals of the subcontract laboratory or per the 
standard methods being used to make each measurement. Any discovered deficiencies with a 
particular instrument will be resolved per the protocol of the laboratory in a timely manner. 
When technical staff operate and maintain technologies undergoing testing, those activities will 
follow directions provided by the technology vendor. Any maintenance required on components 


of the lead paint test kits will be the responsibility of the vendor. 


B7 CALIBRATION/VERIFICATION OF TEST PROCEDURES 


The ICP instrument used for the reference analyses will be calibrated per the standard 
reference method or the standard operating procedures of the analysis laboratory. A calibration 
will be run at the beginning of each set of analyses or at least once daily. The calibration 
coefficient must be at least 0.995 or higher. An independent calibration verification (ICV) 
standard will be run once a day after calibration and a continuous calibration verification (CCV) 
standard will be run at the beginning and end of each sample run. The ICV and CCV must be 
within +10% and +20%, respectively, of known values. If these evaluation criteria are not met, 
analysis must be stopped and recalibration performed. If the recalibration fails, the standards 
must be re-made and/or the equipment must be evaluated. If any component of a lead paint test 
kit requires calibration, the vendor will provide Battelle technical staff with instructions on how 


to properly maintain such components. 


B8 INSPECTION/ACCEPTANCE OF SUPPLIES AND CONSUMABLES 


PEMs used for verification testing will be made prior to the initiation of this test by 
Battelle in accordance with the Revised Plan For Development And Production Of Performance 
Evaluation Materials For Testing Of Test Kits For Lead In Paint under the Environmental 
Technology Verification Program’. The film thickness, homogeneity, and lead levels of the 
paint applied to the PEMs will be verified prior to full-scale PEMs production’. This process 


will test the paint formulation and application to ensure that the desired lead level is being 
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achieved during full PEM production. PEMs will be measured for film thickness using a 
Positector 6000 coating thickness gauge. Subsequently, paint chip samples corresponding to the 
locations of the film thickness measurements will be obtained following ASTM E1729. These 
samples will be shipped to the NLLAP-accredited laboratory for analysis. This process will not 
verify the lead levels of individual PEMs used for verification testing and will take place prior to 
the production of PEMs for this verification test. Details on this process can be found in the 


PEMs development plan’. 


B9 NON-DIRECT MEASUREMENTS 


No non-direct measurements will be used during this verification test. 


B10 DATA MANAGEMENT 


Various types of data will be acquired and recorded electronically or manually by 
Battelle during the verification test. Table 2 summarizes the types of data to be recorded. All data 
and observations for the operation of the test kits will be documented by Battelle technical staff 
on data sheets or in laboratory record books. Results from the subcontract laboratory reference 
instruments will be compiled by the subcontractor’s staff in electronic format and submitted to 
Battelle upon obtaining the results. 

Records received by or generated by any technical staff during the verification test will 
be reviewed by a Battelle staff member within two weeks of generation or receipt, before the 
records are used to calculate, evaluate, or report verification results. If a Battelle staff member 
generated the record, this review will be performed by a Battelle technical staff member involved 
in the verification test, but not the staff member who originally generated the record. The review 
will be documented by the person performing the review by adding his/her initials and date to the 
hard copy of the record being reviewed. In addition, any calculations performed by technical 
staff will be spot-checked by Battelle QA and/or technical staff to ensure that calculations are 
performed correctly. Calculations to be checked include any statistical calculations described in 
this test/QA plan. The data obtained from this verification test will be compiled and reported 
independently for each technology. Results for technologies from different vendors will not be 


compared with each other. 
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Among the QA activities conducted by Battelle QA staff will be an audit of data quality. 
This audit will consist of a review by the Battelle Quality Manager (or his designee) of at least 
10% of the test data. During the course of any such audit, the Battelle Quality Manager will 
inform the technical staff of any findings and any need for immediate corrective action. If serious 
data quality problems exist, the Battelle Quality Manager will request that Battelle’s Verification 
Test Coordinator issue a stop work order. Once the assessment report has been prepared, the 
Verification Test Coordinator will ensure that a response is provided for each adverse finding or 
potential problem, and will implement any necessary follow-up corrective action. The Battelle 


Quality Manager will ensure that follow-up corrective action has been taken. 
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SECTION C 
ASSESSMENT AND OVERSIGHT 


Cl ASSESSMENTS AND RESPONSE ACTIONS 


Every effort will be made in this verification test to anticipate and resolve potential 
problems before the quality of performance is compromised. One of the major objectives of this 
test/QA plan is to establish mechanisms necessary to ensure this. Internal quality control 
measures described in this test/QA plan, which is peer reviewed by a panel of outside experts, 
implemented by the technical staff and monitored by the Verification Test Coordinator, will give 
information on data quality on a day-to-day basis. The responsibility for interpreting the results 
of these checks and resolving any potential problems resides with the Verification Test 
Coordinator. Technical staff have the responsibility to identify problems that could affect data 
quality or the ability to use the data. Any problems that are identified will be reported to the 
Verification Test Coordinator, who will work with the Battelle Quality Manager to resolve any 
issues. Action will be taken to control the problem, identify a solution to the problem, and 
minimize losses and correct data, where possible. Independent of any EPA QA activities, 
Battelle will be responsible for ensuring that the following audits are conducted as part of this 


verification test. 


C1.1 Performance Evaluation Audits 


A Performance Evaluation (PE) audit will be conducted to assess the quality of the 
reference method measurements made in this verification test. The reference method PE audit 
will be performed by supplying an independent, NIST-traceable lead paint standard, to the 
subcontract laboratory. The PE audit samples will be analyzed in the same manner as all other 
samples and the analytical results for the PE audit samples will be compared to the nominal 
concentration. The target criterion for this PE audit is agreement of the analytical result within 
20% of the nominal concentration. If the PE audit result does not meet the target criterion, the PE 
audit will be repeated. If the outlying results persist, the source of error will be investigated and 
corrective action taken as necessary until successful PE audit results are obtained. This audit will 
be performed once at the start of the test, and will be the responsibility of the Verification Test 


Coordinator or designee. 
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C1.2 Technical Systems Audits 


The Battelle Quality Manager or designee will perform a TSA of the actual evaluation of 
the test kits at least once during this verification test. The purpose of this audit is to ensure that 
the verification test is being performed in accordance with the AMS Center and ETV Program 
QMPs"*, this test/QA plan, any published reference methods, and any Standard Operating 
Procedures (SOPs) used. In the TSA, the Battelle Quality Manager, or a designee, may review 
the reference methods used, compare actual test procedures to those specified or referenced in 
this plan, and review data acquisition and handling procedures In the TSA, the Battelle Quality 
Manager will observe testing in progress, observe the reference method sample preparation and 
analysis (when available), inspect documentation, and review technology-specific record books. 
He or she will also check standard certifications and technology data acquisition procedures, and 
may confer with technical staff. A TSA report will be prepared, including a statement of findings 
and the actions taken to address any adverse findings. The EPA Quality Manager will receive a 
copy of Battelle’s TSA report. An initial, draft TSA report, not including verification of 
corrective actions, will be submitted to EPA within 2 weeks of completion of the audit. At 
EPA’s discretion, EPA QA staff may also conduct an independent on-site TSA during the 
verification test. The TSA findings will be communicated to technical staff at the time of the 


audit and documented in a TSA report. 


C1.3 Data Quality Audits 


The Battelle Quality Manager or his designee will audit at least 10% of the verification 
data acquired in the verification test, including any data packages received from the subcontract 
reference laboratory. Data packages will conform to ADQ guidelines provided by EPA. The 
Battelle Quality Manager will trace the data from initial acquisition, through reduction and 
statistical comparisons, to final reporting. All calculations performed on the data undergoing the 
audit will be checked. An initial, draft TSA report, not including verification of corrective 
actions, will be submitted to EPA within 2 weeks of completion of the audit. Data packages 
received from the subcontract laboratory will be briefly reviewed by Battelle for completeness 


and then provided to EPA within three days of receipt of the data package. The packages will 
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not have gone through any data quality audits at this time, as these audits will be ongoing at the 


time of submission of the data to EPA. 


C1.4 QA/QC Reporting 


Each assessment and audit will be documented in accordance with the AMS Center and 


1,6 
ETV Program QMP. The results of the technical systems and data quality audit will be 

submitted to EPA. Initial, draft reports, not including verification of corrective actions, will be 
submitted to EPA within 2 weeks of completion of the audit. Assessment reports will include 


the following: 


e Identification of any adverse findings or potential problems 

e Response to adverse findings or potential problems 

e Recommendations for resolving problems 

e Confirmation that solutions have been implemented and are effective 


e Citation of any noteworthy practices that may be of use to others. 


C2 REPORTS TO MANAGEMENT 


The Battelle Quality Manager, during the course of any assessment or audit, will identify 
to the technical staff performing experimental activities any immediate corrective action that 
should be taken. If serious quality problems exist, the Battelle Quality Manager will notify the 
Battelle Management to authorize a stop work order. Once the assessment report has been 
prepared, the Verification Test Coordinator will ensure that a response is provided for each 
adverse finding or potential problem and will implement any necessary follow-up corrective 
action. The Battelle Quality Manager will ensure that follow-up corrective action has been taken. 
The test/QA plan and final report are reviewed by EPA QA Manager and the EPA ESTE Project 
Officer. Upon final review and approval, both documents will then be posted on the ETV 


website (www.epa.gov/etv). 
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SECTION D 
DATA VALIDATION AND USABILITY 


D1 DATA REVIEW, VALIDATION, AND VERIFICATION REQUIREMENTS 


The key data review requirements for the verification test are stated in Section B10 of 
this test/QA plan. In general, the data review requirements specify that the data generated during 
this test will be reviewed by a Battelle technical staff member within two weeks of data 
generation. The reviewer will be familiar with the technical aspects of the verification test, but 
will not be the person who generated the data. This process will serve both as the data review 
and the data verification, and will ensure that data have been recorded, transmitted, and 
processed properly. Furthermore, this process will ensure that the lead paint test kit data and the 
reference method data are collected under appropriate testing conditions and that the reference 
method data meet the reference method specifications. 

The data validation requirements for this test involve a data quality assessment relative to 
the DQIs and audit acceptance criteria specified for this test. The DQIs listed in Section A 4.2 
and B5 will be used to validate the data quality. The QA audits described within Section C of 
this document, including the performance evaluation audit and data quality audit, are designed to 


validate the data quality. 


D2 VALIDATION AND VERIFICATION METHODS 


Data verification is conducted as part of the data review, as described in Section B10 for 
this test/QA plan. A visual inspection of handwritten data will be conducted to ensure that all 
entries were properly recorded or transcribed, and that any erroneous entries were properly noted 
(i.e., single line through the entry with an error code and the initials of the recorder and date of 
entry). Data manually incorporated into spreadsheets for use in calculations will be checked 
against handwritten data to ensure that transcription errors have not occurred. All calculations 
used to transform the data will be reviewed to ensure the accuracy and the appropriateness of the 
calculations. Calculations performed manually will be reviewed and repeated using a handheld 
calculator or commercial software (e.g., Excel). Calculations performed using standard 
commercial office software (e.g., Excel) will be reviewed by inspecting the equations used in 


calculations and verifying selected calculations by handheld calculator. Calculations performed 
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using specialized commercial software (i.e., for analytical instrumentation) will be reviewed by 
inspection and, when feasible, verified by handheld calculator, or standard commercial software. 

To ensure that the data generated from this test meet the goals of the test, a number of 
data validation procedures will be performed. Section C of this test/QA plan provides a 
description of the validation safeguards employed for this verification test. Data validation and 
verification efforts include the completion of QC activities and the performance of TSA audits as 
described in Section C. The data from this test will be evaluated relative to the measurement 
DQIs described in Section A7.1 and BS of this test/QA plan. Data failing to meet these criteria 
will be flagged in the data set and not used for evaluation of the technologies, unless these 
deviations are accompanied by descriptions of their potential impacts on the data quality. 

A data quality audit will be conducted by the Battelle Quality Manager to ensure that data 
review, verification, and validation procedures were completed, and to assure the overall data 


quality. 


D3 RECONCILIATION WITH USER REQUIREMENTS 


This purpose of this verification test is to evaluate the performance of qualitative spot test 
kits for lead based paint. To meet the requirements of the user community, the data obtained in 
this verification test should include thorough documentation of the performance of each lead 
paint test kit during the verification test. The data review, verification, and validation procedures 
described in previous sections will assure that data meet these requirements, are accurately 
presented in the verification reports generated from this test, and that data not meeting these 
requirements are appropriately flagged and discussed in the verification reports. Additionally, all 
data generated using the reference method, which are used to evaluate technology results during 
the verification test, should meet the QA requirements of any applicable standard operating 
procedures or instrumentation instruction manuals. 

The data from this verification test will be compiled into an ETV verification report. The 
report will be submitted to EPA in Word and Adobe pdf format and subsequently posted on the 
ETV website. This test/QA plan and the resulting ETV verification report(s) will be subjected to 
review by the lead paint test kit vendors, Battelle staff, EPA, and expert peer reviewers. The 
reviews of this test/QA plan will assure that this verification test and the resulting reports meet 


the needs of potential users of the qualitative spot test kits for lead based paint. 
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REVISED PLAN FOR 
DEVELOPMENT AND PRODUCTION OF 
PERFORMANCE EVALUATION MATERIALS FOR 
TESTING OF TEST KITS FOR LEAD IN PAINT UNDER THE 
ENVIRONMENTAL TECHNOLOGY VERIFICATION PROGRAM 


July 2009 
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1. Overview of Problem 


The accurate and efficient identification of lead-based paint in housing is important to the 
Federal government and to private individuals living in residences containing such paints, 
especially when renovation, repair or painting work is planned. Renovation, repair and painting 
(RRP) may disturb painted surfaces and produce a lead exposure hazard. According to a recent 
report by the President’s Task Force on Environmental Health Risks and Safety Risks to 
Children, approximately 24 million U.S. dwellings were at risk for lead-based paint hazards in 
1999. The term lead-based paint means paint or other surface coatings that contain lead at 
contents that “equal or exceed a level of 1.0 milligram per centimeter squared or 0.5 percent by 
weight.” 


The Preamble for the proposed EPA rule for RRP activities mentions the development of an 
improved test kit for paint that has a false negative rate of no more than 5 percent and a false 
positive rate of no more than 10 percent vis-a-vis the federal standards for lead-based paint. The 
Preamble also refers to an EPA evaluation and recognition program for test kits, initially for kits 
that are candidates to meet the goal of a 5 percent false negative rate, and then for kits that are 
candidates to meet the joint goals of a 5 percent false negative rate and a 10 percent false positive 
rate. As stated in the Preamble, test kit performance would have to be “validated by a laboratory 
independent of the kit manufacturer, using ASTM International’s E1828, Standard Practice for 
Evaluating the Performance Characteristics of Qualitative Chemical Test Kits for Lead in Paint 
(Ref. 50) or an equivalent validation method. The instruction for use of any particular kit would 
have to conform to the results of the validation, and the certified renovator must follow the 
manufacturer’s instructions when using the kit.” 


This effort includes the development and production of Performance Evaluation Materials 
(PEMs) for use in the evaluation of the performance characteristics of qualitative test kits for 
lead in paint. These PEMs will be used by independent laboratories for the verification of 
existing and new test kits under the Environmental Technology Verification (ETV) program. 
This effort includes the development and production of as many as 6,600 PEMs measuring 
approximately 3 inches by 3 inches. Note that estimates of the number of test panels needed 
assume that multiple tests can be conducted on each panel. 


2. Study Objectives 


The EPA requires a sufficient number of PEMs to evaluate up to 14 different test kits. Although 
a formal power study was not conducted, it is assumed that this number of panels should be 
sufficient to thoroughly test the kits, given that multiple tests can be conducted on each panel. 
How many tests can be conducted on each panel will depend on the specific requirements of 
each test kit. 


In comparison to the NISTIR 6398 study “Spot Test Kits for Detecting Lead in Household Paint: 
A Laboratory Evaluation”, this effort will include many of the same variables. Table 1 
highlights the differences and similarities to both studies. Neither the NISTIR 6398 study nor 
the current PEM production effort described here claims to produce Certified Reference 
Materials (CRM’s) as defined by ISO Guides 31, 34 and 35. Meeting that certification requires 
storage and transportation stability studies which can take up to 36 months. These PEMs are 
expected to be utilized within 18 months of production. The PEMs produced for this study will 
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each be labeled and individually wrapped and sealed to minimize surface abrasions during 


storage and transport. 


Table 1: NISTIR 6398 vs. PEM Production 


























Category NISTIR 6398 (2000) PEM Production Comments 
(2008) 
Certified Reference |no no See note below 
Materials per ISO 
Guides 31, 34 & 35 
Lead Types Lead Carbonate & Lead | Lead Carbonate & Lead 
Chromate Chromate 
Lead Levels 10 levels (0 up to 3.5) Seven levels (0, 0.3, 0.6, 
(mg/em’) depending on lead type _{ 1.0, 1.4, 2.0, 6.0) 
and Coefficient of 
Variation results after 
production 
Substrates Metal - Mylar and release | Apply directly to wood, | Application directly to the 
paper then glue applied to | metal, drywall and substrates closely simulates 
wood and plaster plaster real-world scenario 
Application methods | Drawdown and roller Spray for zero lead Drawdown allows for 
panels and topcoats; direct contact between 
Drawdown for all others | leaded paint and substrate 
and better consistency of 
film thickness 
Leaded Paint Type | Linseed Oil/ Lead paste _| Linseed Oil Based Historically accurate 
added to alkyd paint linseed oil paint 
formulation 
Fineness of Lead None ASTM D1210 Ensure lead pigment 


Dispersion Test 


agglomerates are 
adequately pulverized and 
sufficiently dispersed 
through paint 





Topcoats 


2 = Alkyd and latex 


2 = Primer and latex 





Verification Study 


Originally 37 different 
paints were produced and 
applied, once analyzed, 
results from only 19 total 
paints were actually 
included in the test 


Verification of lead level 
applied with a given film 
thickness on a small 
number of panels prior 
to full production 








Homogeneity 
Analysis 





After panel production 
was completed a 
Coefficient of Variation 
was determined using ICP 
analysis and a variation of 
less than 20% was 
accepted 





Independent 
homogeneity study 
planned prior to and 
throughout full PEM 
production using ICP 
analysis 








3: Study Design Options 


The EPA specified four main design factors of interest — paint speciation, substrate, paint lead 
level, and overcoat color. These are discussed below along with topcoat color, number of 
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topcoats, and panel size. Table 2 presents a simple design that would yield 468 panels per test 
kit for the ETV evaluation. Across the nine kits originally planned for this adds up to 4,212 
panels and across 14 test kits this adds up to 6,552 panels. This number of PEMs should allow 
for sufficient replicates of each design factor of interest for the ETV tests. 


3.1 Paint types 


A typical interior paint formulation containing either white lead (lead carbonate) or lead 
chromate will be prepared. The binder composition will be linseed oil based. The preparation of 
linseed oil-based paint PEMs is substantially longer due to the increased drying time required for 
this type of chemistry. 


3.2 Substrates 

We will use wood (both poplar and pine), metal (zinc phosphated cold-rolled steel), drywall, and 
plaster (limestone base) applied over drywall. The two types of wood are proposed in order to 
better represent potential substrates across the range of pre-1978 housing. These represent 
typical substrates to which leaded paint was originally applied. 


3.3. Lead Levels 

This plan proposes producing panels at seven lead levels: 0.0 mg/cm’, 0.3 mg/cm’, 0.6 mg/cm’, 
1.0 mg/cm’, 1.4 mg/cm’, 2.0 mg/cm’, and 6.0 mg/cm’. These levels represent two levels 
containing lead below the current lead based paint standard of 1.0 mg/cm”, one level at the 
standard, two levels slightly above the standard, and additional higher level (6.0 mg/cm”) to 
investigate differences in performance at significantly higher lead levels as well as represent 
potential lead levels on exterior surfaces, which are higher than those found on interior walls. 
The level of the current standard, 1.0 mg/cm”, is included to allow for the evaluation results to 
provide a smoother performance evaluation curve. In addition, blank samples with no lead paint 
applied will also be produced for testing and quality control purposes. 


3.4 Paint Colors 

Three colors of topcoats will be applied randomly across the panels. The following paint colors 
will be used: white, reddish-orange, and grayish-black. These colors were chosen based on 
potential for certain colors to cause interferences with the performance of some kits. 


3.5 Layers of paint on top of lead paint (Topcoats) 


Two different layers of paint will be applied over the leaded paint. One will be a primer 
designed for adhesion to linseed oil-based paint and the second coat will be a typical interior 
modern latex paint tinted to one of three colors. The thickness of each topcoat is not a design 
variable. The manufacturers’ recommended application thickness will be used. Two coats at the 
recommended thickness will be applied. 


3.6 Panel Size 
3 inch by 3 inch PEMs will be produced. The assumption is that multiple tests (up to four 
replicates) can be conducted per panel. 
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Table 2: PEMs Produced for ETV Test 
















































































































































































# Samples Produced Per Test Kit by Topcoat Color 9 Test Kits 
Lead Type Lead Level Substrate White | Red-Orange | Grey-Black Total 
Wood-Poplar 2 1 9 5 45 
Wood-Pine 1 2 4 4 36 
Control Blank 0 Metal 3 3 3 9 81 
Drywall 3 3 3 9 81 
Plaster 3 3 3 9 81 
Wood-Poplar 2 1 2 5 45 
Wood-Pine 1 2 1 4 36 
0.3 Metal 3 3 3 9 81 
Drywall 3 3 3 9 81 
Plaster 3 3 3 9 81 
Wood-Poplar 1 2 1 4 36 
Wood-Pine 2 1 2 5 45 
0.6 Metal 3 3 3 9 81 
Drywall 3 3 3 9 81 
= Plaster 3 3 3 9 81 
a: Wood-Poplar 2 1 2 5 45 
<<. Wood-Pine 1 2 1 4 36 
1.0 Metal 3 3 3 9 81 
Lu Drywall 3 3 3 9 81 
P Plaster 3 3 3 9 81 
= White Lead Nese uae : ; 7 
Wood-Pine 2 1 2 5 45 
| 1.4 Metal 3 3 3 9 81 
Drywall 3 3 3 9 81 
YY Plaster 3 3 3 9 81 
© Wood-Poplar 2 1 2 5 45 
Wood-Pine 1 2 1 4 36 
Q 2.0 Metal 3 3 3 9 81 
Drywall 3 3 3 9 81 
Plaster 3 3 3 9 81 
LW Wood-Poplar 1 2 1 4 36 
; Wood-Pine 2 1 2 5 45 
=> 6.0 Metal 3 3 3 9 81 
—| Drywall 3 3 3 9 81 
Plaster 3 3 3 9 81 
7 Wood-Poplar 27 243 
UW 04.06 10 Wood-Pine_| 2 panels per cell for Wood substrates, 4 27 243 
Lead Chromate 1 4 20 60 Metal panels per cell for other substrates (same 54 486 
oY aaa Drywall design as White Lead panels) 5A 486 
Plaster 54 
<x 
fa 4. Production Plan 
Ww 4.1 Materials 
Nine different paint formulations will be produced as dictated by the two lead pigments (lead 
V) carbonate and lead chromate) and the six different lead levels in addition to the zero lead level 
a | control. The formulations will be designed to consistently achieve the lead levels required when 


applied at typical wet film builds. The paint samples will be produced using standard painting 
production procedures in the Battelle laboratories including pre-mixing, media grinding of 
pigment and binder resin, and paint letdown with resin and solvents. This procedure has been 
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used for paint production both in the laboratory and in commercial paint manufacturing for over 
50 years. Battelle staff members completing this task have many years of direct industry 
experience in commercial and laboratory paint production. The paint formulations are shown 
below on Tables 3 and 4. Since the molecular compositions of the two lead pigments are 
different, the formulations have accounted for this by adjusting the load levels. Additional 
formulation changes will be investigated and implemented as necessary during production of the 
paints for the ETV evaluation. 


The lead pigment fineness of dispersion will be monitored throughout the grinding process and 
once a Hegman grind of 5-6 has been achieved (ASTM D1210), the paint will be let-down with 
additional turpentine and Japan drier. 


Table 3: Lead Carbonate Paint Formulations 


0% Zero Lead Paint Formulation | 


ZnO The Carry Co. 1491.75 

Pb CO; American Elements 0.00 

TiO2 DuPont 621.56 
Linseed Oil Recochem Inc. 149.18 
Boiled Linseed Oil Recochem Inc. 14.92 
Turpentine Recochem Inc. 217.55 
Japan Drier Barr 5.04 
2500.00 


0.3% Lead Carbonate Paint Formulation | 


ZnO The Carry Co. 1477.08 

Pb CO3 American Elements 147.71 

TiO2 DuPont 492.36 
Linseed Oil Recochem Inc. 147.71 
Boiled Linseed Oil Recochem Inc. 14.77 
Turpentine Recochem Inc. 215.41 
Japan Drier Barr 4.97 
silica (TS 100) Degussa 25.00 
2525.00 


1.0% Lead Carbonate Paint Formulation 


ZnO The Carry Co. 1388.14 

Pb CO3 American Elements 289.20 

TiO2 DuPont 462.71 

Linseed Oil Recochem Inc. 138.81 

Boiled Linseed Oil Recochem Inc. 13.88 

Turpentine Recochem Inc. 202.44 

Japan Drier Barr 4.82 

silica (TS 100) Degussa 37.50 

2537.50 
This formulation will be used to produce 0.6% and 1.4% lead levels at different coating thickness. 
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Gram wt 


2.0% Lead Carbonate Paint Formulation 
| Materials | Supplier | | 


ZnO 


The Carry Co. 


% by wt. 


1204.04 





Pb CO3 


American Elements 


568.31 





TiO2 


DuPont 


385.29 





Linseed Oil 


Recochem Inc. 


132.00 





Boiled Linseed Oil 


Recochem Inc. 


13.20 





Turpentine 


Recochem Inc. 


192.50 





Japan Drier 


Barr 


4.67 





silica (TS 100) 


Degussa 


37.50 














2537.50 


6.0% Lead Carbonate Paint Formulation 
| Materials | Supplier | byt. 


ZnO 


The Carry Co. 


418.16 





Pb CO3 


American Elements 


1612.23 





TiO2 


DuPont 


139.39 





Linseed Oil 


Recochem Inc. 


127.34 





Boiled Linseed Oil 


Recochem Inc. 


12.73 





Turpentine 


Recochem Inc. 


185.70 





Japan Drier 


Barr 


4.45 





silica (TS 100) 


Degussa 


50.00 














2550.00 
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Table 4: Lead Chromate Paint Formulations 


ZnO The Carry Co. 1500.74 
PbCrO, American Elements 110.05 
TiO2 DuPont 500.25 
Linseed Oil Recochem Inc. 150.07 
Boiled Linseed Oil Recochem Inc. 15.01 
Turpentine Recochem Inc. 218.86 
Japan Drier Barr 5.01 
silica (TS 100) Degussa 17.50 
2517.50 



































ZnO The Carry Co. 1388.14 
PbCrO, American Elements 289.20 
TiO2 DuPont 462.71 
Linseed Oil Recochem Inc. 138.81 
Boiled Linseed Oil Recochem Inc. 13.88 
Turpentine Recochem Inc. 202.44 
Japan Drier Barr 4.82 
silica (TS 100) Degussa 25.00 
2525.00 


This formulation will be used to produce 0.6% and 1.4% lead levels at different coating thickness. 






































ZnO The Carry Co. 1058.12 
PbCrO, American Elements 695.72 
TiO2 DuPont 370.34 
Linseed Oil Recochem Inc. 145.00 
Boiled Linseed Oil Recochem Inc. 14.50 
Turpentine Recochem Inc. 211.46 
Japan Drier Barr 4.85 
silica (TS 100) Degussa 25.00 
2525.00 



































ZnO The Carry Co. 149.69 
PbCrO, American Elements 1946.00 
TiO2 DuPont 49.90 
Linseed Oil Recochem Inc. 136.75 
Boiled Linseed Oil Recochem Inc. 13.68 
Turpentine Recochem Inc. 199.43 
Japan Drier Barr 4.54 
silica (TS 100) Degussa 25.00 
2525.00 
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The applied lead levels will be verified prior to full-scale PEM production via drawdown 
application on quality-controlled metal panels. Dried paint chip samples will be obtained and 
evaluated by an independent NLLAP-accredited laboratory using ICP-AES or equivalent 


analysis. 


Quadrant #1 


Quadrant #2 





Film 
Thickness 
Test 
Area 


ICP 
Lead Level 
Test 
Area 


Quadrant #3 


Film 
Thickness 
Test 
Area 


ICP 
Lead Level 
Test 
Area 


Quadrant #4 





Film 
Thickness 
Test 
Area 





ICP 
Lead Level 
Test 
Area 


Film 
Thickness 
Test 
Area 





ICP 
Lead Level 
Test 
Area 





Figure 2. PEM Lead Level Verification Step Test Map 


This lead level verification study 
will test the formulation and 
application to ensure that the 
desired lead level can be achieved 
during full PEM production. The 
test map in Figure | describes how 
each panel will be divided and 
where each test will be performed. 


One panel of each film thickness will be prepared using the previously manufactured paints. 
After drying, the panel will be measured once in each quadrant for film thickness using a 
Positector 6000 coating thickness gauge. Subsequently, paint chip samples corresponding to the 
locations of the film thickness measurements will be obtained following ASTM E1729, 
“Standard Practice for Field Collection of Dried Paint Samples for Subsequent Lead 
Determination.” These samples will be shipped to the NLLAP-accredited laboratory for 
analysis. The laboratory will prepare the paint chip samples for analysis following ASTM 1656 
and analyze the samples according to the ASTM 1613 method, “Standard Test Method for 
Determination of Lead by Inductively Coupled Plasma Atomic Emission Spectrometry (ICP- 
AES), Flame Atomic Absorption Spectrometry (FAAS), or Graphite Furnace Atomic Absorption 
Spectrometry (GFAAS) Techniques.” Results will be provided electronically along with 
relevant QA/QC data on calibration samples, analytical duplicates, and analytical spikes. 


Since the thickness of the paint film will determine the final lead level per unit area, the purpose 
of this verification step is to identify the approximate film thickness that will result in the 
targeted lead level for each paint. The ICP lead level measurements will be used to determine 
the film thickness that led to the desired lead level. If average lead levels of the nearest 
measurements differ from the target by more than 10 percent for target lead levels of 1.0 mg/cm* 
and higher and 15 percent for target lead levels <1.0 mg/cm”, corresponding adjustments will be 
made to the targeted film thickness measurements prior to the homogeneity testing to follow. 


Once the desired film thicknesses are determined from the verification step, panels will be coated 
for a Homogeneity Study consisting of producing three additional panels at each of the six lead 
levels for both types of lead pigments, obtaining four paint chip samples from each panel (one 
from each quadrant), and conducting ICP-AES analysis on those paint samples to determine 
precise lead measurements. The results from this process will confirm the ability to apply 
consistent and accurate lead levels across each panel. Prior to paint chip sampling, film 
thickness measurements also will be obtained to verify the desired film thickness for each leaded 
paint. Table 5 provides an example of the data that will be collected for the Homogeneity study. 
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Table 5: Lead Level Verification/Homogeneity Study for ETV Panels 


Film Thickness Measurements - 1 per ; 


Level 
as | 7 [a7 
ee ee a Ss Ee (eee 


none 


Oo 
wo 


= 
oO 


White Lead 


; 


= 
& 


Lead 


Chromate Same testing scheme as the White Lead paints 





The ICP lead level measurements obtained will be analyzed to establish accuracy of the lead 
levels in reference to target lead levels as well as the variability across and within panels. 
Variability will be measured using the Coefficient of Variation (i.e., the standard deviation 
divided by the mean). A Coefficient of Variation (CoV) less than 15 percent will be deemed 
acceptable. 


4.2 Facilities 
Battelle laboratories include a walk-in spray booth capable of this type of production as well as 
air handling equipment and monitors to ensure staff safety. 


4.3 Painting Process 


Some substrates, such as wood and drywall will be cut down into 3” by 3” panel dimensions. 
The plaster panels will be prepared by applying plaster to pre-cut drywall panels via hand trowel 
application. All drywall and drywall/plaster panels will require edge treatment to minimize 
dusting of the interior gypsum. This treatment will be performed by spray applying a latex 
primer to the edge surfaces only to encapsulate the inner core. All substrate preparations will 
take place prior to the application of any leaded paints. 


The linseed oil-based paints will be applied via drawdown bar directly to the surface of the 
substrates. The drawdown method, although somewhat less efficient for applying coatings to a 
large number of panels, will allow more precise film builds to be achieved as the application 
device will be set to the previously determined target film build level. The drawdown paint 
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application method will be utilized for both the white lead and lead chromate paints at the three 
specified lead levels. The substrates will be lined up in sets of either two or four, depending on 
the substrate type (2 drywall, 2 plaster. 4 metal or 4 wood panels next to each other). The paint 
will be applied across the two or four PEMs in the set at the same time. The order of application 
for each set will be assigned randomly using a random number generator in MS Excel. For 
example, the order of application could be drywall, metal, wood, wood, plaster, metal, etc. The 
PEMs will be placed on a horizontal board to dry overnight. Using the same randomization 
process, a metal reference panel will be coated in the same fashion (same drawdown bar, same 
paint formulation). Reference panels will be inserted as one of the four metal panels in a set, 
with the location within the set randomly determined. Within Table 6, which provides an 
example drawdown scenario for 56 panels targeted for the same lead paint application and same 
topcoat color, three reference panels are inserted into the application process to verify the lead 
levels applied to surrounding panels. The first and second reference panels are randomly placed 
among the first eight and second eight sets, respectively, while the third reference panel is 
randomly placed among the last seven sets of PEMs. The position of the reference panel within 
the set of metal panels is also randomly determined. 


Table 6. Illustration of Reference Panel Location within Set of 56 Production Panels 
Drawdown Positions 


drywall 


| drywall 
plaster plaster 
plaster plaster 
| drywall 


plaster plaster 
plaster 


| drywall | | 14 
15 
plaster_ | plaster | S| | te 
a a a ae 
| plaster | | | 18 
1 


9 

plaster | plaster | | | 20 | 

drywall _| drywall [| = | || atid 
es er ae 


| drywall | | | 


The reference PEMs will be tested for film thickness during application and for lead level 
verification by ICP analysis after the paint has dried. This test procedure will check that the 
application process is resulting in appropriate lead levels. Metal panels will serve as the 
reference panels since the metal panels yield the most accurate measurements of film thickness 
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and lead levels. Previous work found that the metal panels provided the best film thickness 
measurements and also provided a better surface for extracting a complete paint chip sample for 
ICP analysis. Despite the use of metal reference panels, the lead levels and paint thickness on 
the reference panels will be representative of the coatings applied to all wood, drywall, plaster, 
and metal substrate panels that will be randomly scattered throughout each area. 


Once reference panel results have undergone review and are determined to have met all target 
specifications, the PEMs will then be mounted onto 4 foot by 8 foot drying racks for the 
application of topcoat layers and conditioned in constant temperature and humidity rooms to 
ensure consistent curing for all samples. All substrates will be attached to the drying racks for 
painting using double stick tape. Following application of the topcoat layers, the PEMs will be 
removed for packaging and shipping. Packaging will involve attaching a label to each panel, 
covering the panel with a chem-wipe for protection, inserting the panel in a plastic, sealable bag, 
and attaching a matching label to the exterior of the bag. Separate boxes will be prepared 
containing full sets of PEMs. 


5. Quality Assurance/Quality Control 


Lead levels will be verified prior to full panel production by an independent NLLAP-accredited 
laboratory. This detailed testing of initial batches of samples during the PEMs verification step 
will establish that specified lead levels can be achieved. As part of the verification step, ICP 
analysis will also be conducted in multiple locations on initial batches of panels to confirm lead 
levels and homogeneity across panels. During production application of the leaded paints, a 
subset of reference panels for each set of PEMs (unique lead level, lead pigment combinations) 
will be tested to monitor the paint application process and ensure that targeted lead levels are 
achieved with acceptable variability. As noted above, the average lead level across the reference 
panels should be within 10% of the targeted value for panels at 1.0 mg/cm? and higher and 
within 15% for panels below 1.0 mg/cm”. The tolerance for an acceptable level of variability in 
film thickness and lead level is <15% Coefficient of Variation (CoV). One reference panel will 
be inserted into the production process for every 18 to 20 production panels. 


The laboratory will be required to provide QA/QC data from each batch of paint samples 
analyzed on calibration samples, analytical duplicates, and analytical spikes. Percent recovery 
for analytical spike samples must be within 20 percent of the actual value. The relative percent 
difference for analytical duplicates (i.e., the absolute value of the difference between the two 
samples divided by their average) must be less than 20 percent. If any QC samples are found to 
be outside of these tolerance limits, the laboratory will be asked to reanalyze the affected batch 
of samples. 


6. Environmental Safety and Health Issues 


Battelle has a safety and health plan related to producing lead-based paint and PEMs coated with 
these paints. The plan was approved by appropriate environmental safety and health personnel. 
Environmental monitoring during paint mixing and spraying activities determined that lead 
exposure levels were below OSHA standards. During development of the PEMs for the ETV 
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evaluation, Battelle staff will continue to comply with the safety and health plan. Some of the 
components of the safety plan include: 
e All staff (and any visitors) will need hazard communication training on lead. 
e Baseline and post-work blood-lead levels will be obtained on those that will be 
conducting the paint mixing and spray painting. 
e Respirators will be used during leaded paint production and the spray application 
operation by staff that have undergone the required physical, training and fit test 
e The interior of the spray booth will be covered with plastic or other material that can be 
easily removed and disposed as hazardous waste. 
e The area in front of the booth will be set up as a change out area where the coveralls, etc, 
can be removed without spreading lead outside of the area. 


Warning signs at the paint booth door and restricting access will be posted. 
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Summary of Homogeneity and Lead Levels Analyses for 
PEMs Development 
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Table A-1. Results from Final Homogeneity Testing for each set of ETV PEMs 















No Lead og tool asl azo 
oss 098 | 7 
| 60 | || 14.2| 
a) 
| oe | 2] 98] 1 


White 
Lead 


or 
Chromate 


* Coefficient of a (Standard Deviation/Mean x x 100) 








3.3 





Table A-2. Reference Panel Results from Final Production for each set of ETV PEMs 
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Appendix B 


Standard Operating Procedure (SOP) 
For Collection of Dried Paint Samples for Lead Determination 
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I, Scope 

This Standard Operating Procedure (SOP) describes the manner to which dried paint 
samples should be collected for the determination of lead on substrates such as metal, 
wood, plaster and drywall. 

Il. Purpose 
The purpose of this SOP is to create a reproducible sampling technique that minimizes 
variability between samples and to accurately represent a lead level for a specific 
substrate. 


Til. References 


A. ASTM E 1729-05 Standard Practice for Field Collection of Dried Paint Samples 
for Subsequent Lead Determination. 


B. SOP AMA I-006-02 Standard Operating Procedure for the Safe Handling of Lead, 
Lead-Spiked Paint, and Lead-Spiked Samples 


C. OSHA Standards for Lead-1910.1025 
D. Battelle Safety and Industrial Hygiene Program Plan (HESHQ-SIH-PP-005) 


E. ETV ESTE Test/QA Plan For The Verification Of Qualitative Spot Test Kits For 
Lead In Paint 


IV. Definitions 


A. Paint collection tray-any clean, dry, lead-free container for use in catching paint 
scrapings. 


B. PPE-Personal protective equipment 


C. Lead worker-Training for any person physically handling lead powders or lead- 
containing materials such as paints. 


Vi Procedures 
A. Equipment and Materials 
i, PPE 


1. Disposable Coveralls/Scrubs 

2. Disposable Gloves 

3. Disposable Shoe Covers 

4. Safety Glasses with Side Shields 
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ii. Equipment 


glass vials and lids 

labels 

tape 

Sampling template — one inch square reusable aluminum or steel 
template with accurately known dimensions 

disposable wipes-for cleaning off sampling equipment 
sharp-edged knives/blades/chisels for cutting and scraping 
tweezers 

Paint collection tray 

small brushes-for brushing off powder to weighing paper for 
transfer 


a ae a 


SOOO ON ON 


B. Lead workers with full PPE in a dedicated lead hazard area with lead disposal will 
be used to minimize exposure to other persons or areas. 
Label vial for collection using the number on the back of the panel 
Place the 1 inch square template as close to the tested area as possible, without 
including any disturbed areas. 
1. There are two types of templates, a one inch square and a punch-out of 
one inch square. See Picture 1. 


gO 





Picture 1: Templates 


E. Metals 
i. While holding the template in place, score an outline of the area using a 
sharp edge. 
il. Remove square template and clean the template with a wipe. Set aside 
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ili. Using tweezers, remove square latex topcoat and place in vial. See 
Picture 2. 

iv. Using a blade, scrape the lead off the exposed one inch area and brush on 
to a collection tray. Make sure to brush all remaining lead and do not 
disturb adjacent area. See Picture 3. 

v. Put contents of tray into the same vial, close lid tightly and label 
appropriately. 





- 
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Picture 2: Metal with latex topcoat removed 





Picture 3: Metal with lead scraped and brushed on to collection tray 
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F. Drywall 
i. While holding the template in place, score an outline of the area using a 
sharp edge. 


li. Remove template and clean template with a wipe, set aside. 
ili. Using tweezers remove latex topcoat. See Picture 4 
iv. Using a blade, scrape the lead off the exposed one inch area, minimizing 
any drywall paper within the area. 
v. Brush all remaining lead off the panel onto the collection tray without 
disturbing the adjacent area. 
vi. Place all material into a vial, close tightly and label properly. 





Picture 4: Drywall with latex topcoat removed 


G. Plaster 
i. While holding the template in place, score an outline of the area using a 
sharp edge. 
li. Remove template and clean template with a wipe, set aside. 
iii. Using tweezers remove latex topcoat. See Picture 5 
iv. Using a blade, scrape the lead off the exposed one inch area, minimizing 
the collection of any plaster with the sample. 
v. Brush on to a collection tray without disturbing the adjacent area. See 
Picture 6 
vi. Place all material into a vial, close tightly and label appropriately 
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PUUIN 7S JOIN GUL in ed Wy Vi 


Picture 5: Latex topcoat removed from plaster sample 





H. Pine and Poplar 
i. While holding the template in place, score an outline of the area using a 
sharp edge. 
li. Remove template and clean template with a wipe, set aside. 
ii. Using tweezers remove latex topcoat. See Picture 7 
iv. Using a blade, scrape the lead off the exposed one inch area, minimizing 
the collection of any wood with the sample. 
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VI. 
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v. Brush on to a collection tray without disturbing the adjacent area. See 
Picture 7 
vi. Place all material into a vial and close tightly 












‘TAT 


Facial 6 a —— : 
Picture 7: Latex topcoat removed from wood panel 


Safety 


Py 


Q 7 


Project staff who work with lead or lead products must complete lead worker 
training. Visitors are not permitted in the lead contaminated areas without prior 
approvals and proof of required medical surveillance and training. 

Staff will be on a Battelle Medical Monitoring Program, and will have had an 
annual physical, etc., as required for working with lead, and will be approved to 
work on projects using lead 

Staff working with lead will be kept to the minimum number required to do the 
project. 


. Clothing or equipment will not be blown, shaken, etc. to remove dust. 


Waste (such as filters, liners, PPE, wipes, ETC.) will be disposed of as per 
Battelle policies/procedures for lead contaminated items. 
MSDS for Lead and Paint used in projects are in Room 5148 


. Battelle has strict policies that no food or beverages be present or consumed, nor 


any cosmetics applied within the laboratory area. Contact lenses will not be 
handled in or around the area 


. Any known or suspected exposure to lead outside of the engineering controls will 


be reported to Battelle Health Services or Safety Health/Emergency Response 
(SH/ER) representative as soon as possible 

Refer to Battelle Safety and Industrial Hygiene Program Plan (ESHQ-SIH-PP- 
005) for additional guidance 
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Vil. Shipping 

A. Lead-containing painted samples will be analyzed further at a different location 

B. Samples will be protected from possibly contaminating the shipping route by 
enclosing the vials in an additional plastic bag and securing the bag with tape. 

C. Sample custody will be documented for the shipping and analysis of PEMs to the 
subcontract laboratory using standard chain-of-custody (COC) forms provided by 
the laboratory performing the analysis. 

D. The original sample COC forms will accompany the samples; the shipper will 
keep a copy. 

E. Copies of all COC forms will be delivered to the Verification Test Coordinator 
and maintained with the verification test records. 
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